Uncertain Trees: Dealing with Uncertain Inputs in Regression Trees

Tree-based ensemble methods, as Random Forests and Gradient Boosted Trees, have been successfully used for regression in many applications and research studies. Furthermore, these methods have been extended in order to deal with uncertainty in the output variable, using for example a quantile loss in Random Forests (Meinshausen, 2006). To the best of our knowledge, no extension has been provided yet for dealing with uncertainties in the input variables, even though such uncertainties are common in practical situations. We propose here such an extension by showing how standard regression trees optimizing a quadratic loss can be adapted and learned while taking into account the uncertainties in the inputs. By doing so, one no longer assumes that an observation lies into a single region of the regression tree, but rather that it belongs to each region with a certain probability. Experiments conducted on several data sets illustrate the good behavior of the proposed extension.Comment: 9 page

Multiclass SVM with graph path coding regularization for face classification

International audienceWe consider the problem of learning graphs in a sparse multiclass support vector machines framework. For such a problem, sparse graph penalty is useful to select the significant features and interpret the results. Classical ℓ 1 -norm learns a sparse solution without considering the structure between the features. In this paper, a structural knowledge is encoded as directed acyclic graph and a graph path penalty is incorporated to multiclass SVM. The learned classifiers not only improve the performance, but also help in the interpretation of the learned features. The performance of the proposed method highly depends on an initialization graph. Two generic ways to initialize the graph between the features are considered: one is built from similarities while the other one uses Graphical Lasso. The experiments of face classification task on Extended YaleB database verify that: i) graph regularization with multiclass SVM improves the performance and also leads to a more sparse solution compared to ℓ 1 -nor

Uncertain Trees: Dealing with Uncertain Inputs in Regression Trees

Publication arXiv, travail de recherche postdoctoral sur les arbres de décision probabilistesTree-based ensemble methods, as Random Forests and Gradient Boosted Trees, have been successfully used for regression in many applications and research studies. Furthermore, these methods have been extended in order to deal with uncertainty in the output variable, using for example a quantile loss in Random Forests [16]. To the best of our knowledge, no extension has been provided yet for dealing with uncertainties in the input variables, even though such uncertainties are common in practical situations. We propose here such an extension by showing how standard regression trees optimizing a quadratic loss can be adapted and learned while taking into account the uncertainties in the inputs. By doing so, one no longer assumes that an observation lies into a single region of the regression tree, but rather that it belongs to each region with a certain probability. Experiments conducted on several data sets illustrate the good behavior of the proposed extension